Handwritten data input device and method, and authenticating device and method

ABSTRACT

There are provided a device and method for enabling entry of handwritten data, such as a signature, without use of an electronic tablet. In step S 2 - 1 , a user affixes a signature by means of moving a pen device  30  in midair. In step S 2 - 2 , camera means  20  photographs the pen device  30  and outputs the resultant image data. In step S 2 - 3 , coordinate computation means  22  determines coordinates of a pen tip of the pen means  30  on a screen on the basis of the image data (motion picture data), thereby determining signature data. In step S 2 - 4 , comparison verification means  40  compares and verifies the thus-determined signature data with reference data. If a similarity exists between the signature data and the reference data, there is output an authentication result showing “authentication of verification.” Thus, personal identification can be effected on the basis of the signature data written by the user in midair.

FIELD OF THE INVENTION

The invention relates to an apparatus and method for enabling entry ofhandwritten data, such as a signature. Particularly, the inventionrelates to an apparatus and method for enabling entry of a signatureused for personal identification.

BACKGROUND ART

A personal identification method utilizing signature data is widelyknown as one personal identification method for identifying a person.

This method usually requires an input apparatus called an electronictablet for enabling entry of signature data. Moreover, the electronictablet must have a writing surface of sufficient size for a user towrite a signature. When the user writes a signature on the writingsurface, the electronic table acquires time-sequence data pertaining tocoordinates of the pen tip, writing pressure of a pen, or the like areacquired by the electronic tablet. The thus-acquired data are utilizedas signature data for personal identification.

DISCLOSURE OF THE INVENTION

As mentioned above, the electronic tablet must have a certain size andarea in light of the nature thereof, thus imposing a limitation onminiaturization (or a reduction in the area of) the tablet.

Incorporating such an electronic tablet into a compact electronicdevice, such as a portable cellular phone, is usually difficult. Thereis a high probability of forceful incorporation of an electronic tabletinto a portable cellular phone hindering miniaturization of the portablecellular phone.

When signature data are input into a common electronic device other thana portable cellular phone, the electronic device must employ thepreviously-described electronic tablet having a predetermined area. Evenin the case of a common electronic device, use of an electronic tabletis a contributing factor to imposing given limitations on the design ofthe electronic device.

The invention has been conceived in view of the problem, and a firstobject of the invention is to provide an apparatus and method forenabling entry of handwritten data, such as a signature, without use ofan electronic tablet. Further, the invention aims at providing anapparatus and method which enable personal identification based onsignature data by means of utilization of the apparatus and method.

As will be described later, in order to achieve the object, theinventors of the invention propose an apparatus and method which enableacquisition of signature data from image data by means of photographinga signing action to be performed in midair by a user through use of apen. Here, the expression “signing action” means that a user moveshis/her hand as if affixing a signature in midair.

In the case of entry of signature data by way of an electronic tablet,data pertaining to a writing pressure are often detected, and the dataalso constitute a portion of signature data. In contrast, theapparatus/method which enables acquisition of signature data by means ofphotographing a signing action performed in midair can obtaintime-series data pertaining to two-dimensional coordinates on a screenbut encounters difficulty in obtaining third data, such as a writingpressure.

Therefore, related-art signature matching is effected by means ofmatching three-dimensional data consisting of three types of data, suchas a vertical coordinate, a horizontal coordinate, and a writingpressure. In contrast, according to the technique involving performanceof signing action in midair, there is effected matching oftwo-dimensional data consisting of vertical and horizontal coordinates.Consequently, the technique is presumed to be lower in precision thanrelated-art personal identification.

The invention has been conceived in light of the problems, and a secondobject of the invention is to provide signature data of higher accuracywhen signature data are obtained by means of photographing a signingaction to be performed in midair.

To solve the problem, the invention provides a handwritten data inputapparatus which acquires handwritten data written in midair by a usermoving a pen device in midair, the apparatus comprising:

camera means for acquiring motion picture data pertaining to the pendevice moved by the user; and

coordinate computation means which determines time-series datapertaining to coordinates of a pen tip on a screen of the motion picturedata, on the basis of motion picture data pertaining to the pen deviceacquired by the camera means.

By means of such a configuration, handwritten data pertaining to acharacter or drawing written in midair can be acquired as time-seriesdata pertaining to coordinates.

The handwritten data input apparatus of the invention is furthercharacterized in that the pen device has illumination means, and thecoordinate computation means computes coordinates of the illuminationmeans on the screen.

By means of such a configuration, determination of coordinates on ascreen is facilitated by the light originating from the illuminationmeans.

The handwritten data input apparatus of the invention is furthercharacterized in that the illumination means of the pen device isswitchable between an illuminated state and an extinguished state, andthe coordinate computation means computes coordinates of theillumination means on the screen from when the illumination has enteredan illuminated state until the illumination means enters an extinguishedstate.

By means of such a configuration, the user can explicitly show the startand end of handwritten data.

The handwritten data input apparatus of the invention is furthercharacterized in that the coordinate computation means computescoordinates of the pen device on the screen from a start time in whichthe pen device has started moving on the screen from a stationary statelonger than a predetermined first period until a stop time in which thepen device stops moving as a result of having entered a stationary statelonger than a predetermined second period.

By means of such a configuration, the start and end of handwritten datacan be explicitly indicated even when the pen device is not equippedwith illumination means.

The handwritten data input apparatus of the invention is furthercharacterized in that the coordinate computation means determines thepen device to be at rest when a change in coordinates of the pen deviceis smaller than a predetermined level.

In reality, there are considered to be many cases where, when a personattempts to lay a pen device to rest in midair, slight fluctuationsarise in the pen device. Therefore, in actuality, movements smaller thana predetermined level are preferably determined to represent thestationary state.

The handwritten data input apparatus of the invention is furthercharacterized in that the coordinate computation means informs the userof commencement of input of handwritten data when the pen device isdetermined to be at rest for a period longer than the first period andinforms the user of termination of input of handwritten data when thepen device is determined to be at rest for a period longer than thesecond period.

By means of such a configuration, the user can readily ascertain atiming at which signing action is to be started and ascertain whether ornot the apparatus has recognized completion of the signing action.

The handwritten data input apparatus of the invention is furthercharacterized in that the coordinate computation means informs the userof commencement of input of handwritten data by means of sound orinforms termination of input of handwritten data by means of sound.

By means of such a configuration, the user can readily ascertaincommencement of input of handwritten data by means of sound andascertain whether or not the apparatus has recognized completion of thesigning action.

The handwritten data input apparatus of the invention is furthercharacterized in that the coordinate computation means recognizes astart time and end time of computation of handwritten data by means ofexternal sounds.

By means of such a configuration, the start time/end time are explicitlyinstructed by sounds.

The handwritten data input apparatus of the invention is furthercharacterized in that the sounds are the voice of the user or sounds ofa pen switch provided on the pen device.

By means of such a configuration, the user can clearly show the starttime/end time of handwritten data through use of his/her own voice orsounds of a pen switch.

The handwritten data input apparatus according to the invention furthercomprises display means for displaying the time-series data pertainingto coordinates.

By means of such a configuration, the user can visually ascertaintime-series data pertaining to handwritten data.

The invention also provides a personal identification apparatus equippedwith the above-described handwritten data input apparatus, the personalidentification apparatus comprising:

storage means in which are stored beforehand reference data serving assignature data pertaining to an authorized person; and

comparison verification means which verifies the user's signature datainput as handwritten data by the handwritten data input apparatusagainst the reference data and outputs authentication of identificationwhen the handwritten data and the signature data are determined to be asignature affixed by the same person.

By means of such a configuration, there is obtained a personalidentification apparatus having the same function as those of the piecesof handwritten data input apparatus, and personal identification can beperformed by use of signature data written in midair.

In order to solve the problem, the invention provides a handwritten datainput method by which handwritten data written in midair are acquired bymeans of a user moving a pen device in midair, the method comprising:

an acquisition step of acquiring motion picture data pertaining to a pendevice moved by the user; and

a coordinate computation step of determining time-series data pertainingto coordinates of a pen tip on a screen of the motion picture data, onthe basis of the acquired motion picture data pertaining to the pendevice.

By means of such a configuration, handwritten data pertaining tocharacters or graphics written in midair can be acquired as time-seriesdata pertaining to coordinates.

The handwritten data input method of the invention is furthercharacterized in that the pen device has illumination means, andcoordinates of the illumination means on the screen are computed in thecoordinate computation step.

By means of such a configuration, determination of coordinates on ascreen is facilitated by means of light originating from illuminationmeans.

The handwritten data input method of the invention is furthercharacterized in that the illumination means of the pen device isswitchable between an illuminated state and extinguished state, and, inthe coordinate computation step, there are computed coordinates of theillumination means on the screen from when the illumination means hasentered an illuminated state until the illumination means enters anextinguished state.

By means of such a configuration, the user can explicitly show the startand end of handwritten data.

The handwritten data input method of the invention is furthercharacterized in that, in the coordinate computation step, there arecomputed coordinates of the pen device on the screen from a start timein which the pen device has started moving on the screen from astationary state longer than a predetermined first period until a stoptime in which the pen device stops moving as a result of having entereda stationary state longer than a predetermined second period.

By means of such a configuration, the start and end of and written datacan be explicitly indicated even when the pen device is not providedwith illumination means.

The handwritten data input method of the invention is furthercharacterized in that, in the coordinate computation step, the pendevice is determined to be at rest when a change in coordinates of thepen device is smaller than a predetermined level.

In reality, there are considered to be many cases where, when a personattempts to lay a pen device at rest in midair, slight fluctuationsarise in the pen device. Therefore, in actuality, movements smaller thana predetermined level are preferably determined to represent thestationary state.

The handwritten data input method is further characterized in that, inthe coordinate computation step, the user is informed of commencement ofinput of handwritten data when the pen device is determined to be atrest for a period longer than the first period and is informed oftermination of input of handwritten data when the pen device isdetermined to be at rest for a period longer than the second period.

By means of such a configuration, the user can readily ascertain atiming at which signing action is to be started and ascertain whether ornot the apparatus has recognized completion of the signing action.

The handwritten data input method of the invention is furthercharacterized in that, in the coordinate computation step, the user isinformed of commencement of input of handwritten data by means of soundor informs termination of input of handwritten data by means of sound.

By means of such a configuration, the user can readily ascertaincommencement of input of handwritten data by means of sound andascertain whether or not the apparatus has recognized completion of thesigning action.

The handwritten data input method of the invention is furthercharacterized in that, in the coordinate computation step, a start timeand end time of computation of handwritten data are recognized by meansof external sounds.

By means of such a configuration, the start time/end time are explicitlyinstructed by sounds.

The handwritten data input method is further characterized in that thesounds are the voice of the user or sounds of a pen switch provided onthe pen device.

By means of such a configuration, the user can clearly show the starttime/end time of handwritten data through use of his/her own voice orsounds of a pen switch.

The invention also provides a personal identification method includingthe above-described handwritten data input method, the personalidentification method comprising:

a comparison verification step in which the user's signature data inputas handwritten data by the handwritten data input method are verifiedagainst and compared with the reference data which are stored beforehandand serve as signature data pertaining to an authorized user, andauthentication of identification is output when the handwritten data andthe signature data are determined to be a signature affixed by the sameperson.

By means of such a configuration, there is obtained a personalidentification apparatus having the same operation as that of the piecesof handwritten data input apparatus. Particularly, there can be acquiredsignature data written in midair (in three-dimensional space), and thesignature data can be utilized for personal identification.

The invention provides a handwritten data input apparatus which acquireshandwritten data written on a midair plane parallel to the surface of apredetermined flat plate by means of a user moving a pen device alongthe surface of the flat plate, the apparatus comprising:

camera means for acquiring motion picture data pertaining to a pendevice moved by the user; and

coordinate computation means which determines time-series datapertaining to coordinates of a pen tip on a screen of the motion picturedata, on the basis of motion picture data pertaining to the pen deviceacquired by the camera means.

By means of such a configuration, the user can write handwritten data ona flat plate, and the flat plate can be utilized as a guide at the timeof drawing. A transparent plate; e.g., a flat glass plate, is preferableas a flat plate. Further, photographing the handwritten data from anangle oriented to a surface opposite the surface on which handwrittendata are written is also preferable.

The invention also provides a handwritten data input apparatus foracquiring handwritten data written in midair by means of a user moving afinger in midair, comprising:

camera means for acquiring motion picture data pertaining to the fingermoved by the user; and

coordinate computation means for determining time-series data pertainingto coordinates of the finger on a screen of the motion picture data, onthe basis of the motion picture data pertaining to the finger acquiredby the camera means.

By means of such a configuration, the user can enter handwritten datawritten with his/her finger without use of a pen device.

The handwritten data input apparatus of the invention is furthercharacterized in that the finger is provided with a marker, and thecoordinate computation means determines time-series data pertaining tocoordinates of the marker of the finger.

By means of such a configuration, computation of coordinates of a fingeris facilitated by a marker.

The invention also provides a handwritten data input method foracquiring handwritten data written in midair by means of a user moving afinger in midair, comprising:

an acquisition step of acquiring motion picture data pertaining to thefinger moved by the user; and

a coordinate computation step of determining time-series data pertainingto coordinates of the finger on a screen of the motion picture data, onthe basis of the acquired motion picture data pertaining to the finger.

By means of such a configuration, coordinates of a finger can becomputed readily with a marker.

The handwritten data input method of the invention is furthercharacterized in that the finger is provided with a marker, and, in thecoordinate computation step, time-series data pertaining to coordinatesof the marker of the finger are determined.

By means of such a configuration, computation of coordinates of a fingeris facilitated by a marker.

The handwritten data input apparatus is further characterized in thatthe illumination means emits modulated light, and the coordinatecomputation means computes coordinates of a portion of the motionpicture data corresponding to the modulated light.

The handwritten data input method is further characterized in that theillumination means emits modulated light, and in the coordinatecomputation step, computation coordinates of a portion of the motionpicture data corresponding to the modulated light is carried out.

By means of the configuration of the invention, coordinates of a pendevice (or illumination means) can be determined readily by means ofselecting only modulated light.

There will now be described means of the invention for achieving asecond object.

In order to solve the problem, the invention provides a handwritten datainput apparatus which acquires handwritten data written in midair bymeans of a user moving a pen device in midair, the apparatus comprising:

first camera means for acquiring first motion picture data pertaining tothe pen device moved by the user;

second camera means for acquiring second motion picture data pertainingto the pen device from an angle different from that of the first camerameans; and

coordinate computation means for determining three-dimensionalcoordinates of the pen device in midair (in a three-dimensional space)on the basis of the sets of first and second motion picture data anddetermining time-series data pertaining to the coordinates.

By means of such a configuration, handwritten data pertaining tocharacters or graphics can be acquired as time-series data pertaining tothree-dimensional coordinates.

The handwritten data input apparatus of the invention is furthercharacterized in that the pen device has illumination means, and thecoordinate computation means computes coordinates of the illuminationmeans on the screen.

By means of such a configuration, coordinates can be determined by meansof light originating from the illumination means.

The handwritten data input apparatus of the invention is furthercharacterized in that the illumination means of the pen device isswitchable between an illuminated state and extinguished state, and thecoordinate computation means computes coordinates of the illuminationmeans on the screen from when the illumination has entered anilluminated state until the illumination means enters an extinguishedstate.

By means of the configuration, the user can explicitly show the startand end of handwritten data.

The handwritten data input apparatus of the invention is furthercharacterized in that the coordinate computation means computescoordinates of the pen device on the screen from a start time in whichthe pen device has started moving on the screen from a stationary statelonger than a predetermined first period until a stop time in which thepen device stops moving as a result of having entered a stationary statelonger than a predetermined second period.

By means of such a configuration, the start and end of handwritten datacan be explicitly indicated even when the pen device is not equippedwith illumination means.

The handwritten data input apparatus of the invention is furthercharacterized in that the coordinate computation means determines thepen device to be at rest when a change in coordinates of the pen deviceis smaller than a predetermined level.

In reality, there are considered to be many cases where, when a personattempts to lay a pen device at rest in midair, slight fluctuationsarise in the pen device. Therefore, in actuality, movements smaller thana predetermined level are preferably determined to represent thestationary state.

The handwritten data input apparatus is further characterized in thatthe coordinate computation means informs the user of commencement ofinput of handwritten data when the pen device is determined to be atrest for a period longer than the first period and informs the user oftermination of input of handwritten data when the pen device isdetermined to be at rest for a period longer than the second period.

By means of such a configuration, the user can readily ascertain atiming at which signing action is to be started and ascertain whether ornot the apparatus has recognized termination of the signing action.

The handwritten data input apparatus of the apparatus is furthercharacterized in that the coordinate computation means informs the userof commencement of input of handwritten data by means of sound orinforms termination of input of handwritten data by means of sound.

By means of such a configuration, the user can readily ascertaincommencement of input of handwritten data by means of sound andascertain whether or not the apparatus has recognized completion of thesigning action.

The handwritten data input apparatus is further characterized in thatthe coordinate computation means recognizes a start time and end time ofcomputation of handwritten data by means of external sounds.

By means of such a configuration, the start time/end time are explicitlyinstructed by sounds.

The handwritten data input apparatus of the invention is furthercharacterized in that the sounds are the voice of the user or sounds ofa pen switch provided on the pen device.

By means of such a configuration, the user can clearly show the starttime/end time of handwritten data through use of his/her own voice orsounds of a pen switch.

The handwritten data input apparatus of the invention is furthercharacterized by further comprising display means for displaying thetime-series data pertaining to coordinates.

By means of such a configuration, the user can visually ascertaintime-series data pertaining to handwritten data.

The invention also provides a personal identification apparatus equippedwith the handwritten data input apparatus, the personal identificationapparatus comprising:

storage means in which are stored beforehand reference data serving assignature data pertaining to an authorized person; and

comparison verification means which verifies the user's signature datainput as handwritten data by the handwritten data input apparatusagainst the reference data by means of comparison and outputsauthentication of identification when the handwritten data and thesignature data are determined to be a signature affixed by the sameperson.

By means of such a configuration, there is obtained a personalidentification apparatus having the same operation as that of the piecesof handwritten data input apparatus. Particularly, since signature datain the form of three-dimensional data are utilized, highly accuratepersonal identification can be achieved.

To solve the problem the invention also provides a handwritten datainput method by which handwritten data written in midair are acquired bymeans of a user moving a pen device in midair, the method comprising:

a first acquisition step of acquiring first motion picture datapertaining to a pen device moved by the user from a first angle;

a second acquisition step of acquiring second motion picture datapertaining to a pen device moved by the user from a second angledifferent from the first angle; and

a coordinate computation step of determining three-dimensionalcoordinates of the pen device in midair (i.e., in a three-dimensionalspace) on the basis of the acquired sets of first and second motionpicture data pertaining to the pen device and determining time-seriesdata pertaining to the coordinates.

By means of such a configuration, handwritten data pertaining tocharacters and graphics can be acquired in the form of time-series datapertaining to three-dimensional coordinates.

The handwritten data input method of the invention is furthercharacterized in that the pen device has illumination means, andcoordinates of the illumination means on the screen are computed in thecoordinate computation step.

By means of the configuration, three-dimensional coordinates inthree-dimensional space can be determined readily by means of lightoriginating from the illumination means.

The handwritten data input method of the invention is furthercharacterized in that the illumination means of the pen device isswitchable between an illuminated state and extinguished state, and, inthe coordinate computation step, there are computed coordinates of theillumination means on the screen from when the illumination has enteredan illuminated state until the illumination means enters an extinguishedstate.

By means of such a configuration, the start time/end time are explicitlyinstructed by sounds.

The handwritten data input method of the invention is furthercharacterized in that there are computed, in the coordinate computationstep, three-dimensional coordinates of the pen device inthree-dimensional space from a start time in which the pen device hasstarted moving on the screen from a stationary state longer than apredetermined first period until a stop time in which the pen devicestops moving as a result of having entered a stationary state longerthan a predetermined second period.

By means of such a configuration, the start and end of handwritten datacan be explicitly indicated even when the pen device is not equippedwith illumination means.

The handwritten data input method of the invention is furthercharacterized in that, in the coordinate computation step, the pendevice is determined to be at rest when a change in coordinates of thepen device is smaller than a predetermined level.

In reality, there are considered to be many cases where, when a personattempts to lay a pen device at rest in midair, slight fluctuationsarise in the pen device. Therefore, in actuality, movements smaller thana predetermined level are preferably determined to represent thestationary state.

The handwritten data input method of the invention is furthercharacterized in that, in the coordinate computation step, the user isinformed of commencement of input of handwritten data when the pendevice is determined to be at rest for a period longer than the firstperiod and is informed of termination of input of handwritten data whenthe pen device is determined to be at rest for a period longer than thesecond period.

By means of such a configuration, the user can readily ascertain atiming at which signing action is to be started and ascertain whether ornot the apparatus has recognized termination of the signing action.

The handwritten data input method of the invention is furthercharacterized in that, in the coordinate computation step, the user isinformed of commencement of input of handwritten data by means of soundor informs termination of input of handwritten data by means of sound.

By means of such a configuration, the user can readily ascertaincommencement of input of handwritten data by means of sound andascertain whether or not the apparatus has recognized completion of thesigning action.

The handwritten data input method of the invention is furthercharacterized in that, in the coordinate computation step, a start andend time of computation of handwritten data are recognized by means ofexternal sounds.

By means of such a configuration, the start time/end time are explicitlyinstructed by sounds.

The handwritten data input method of the invention is furthercharacterized in that the sounds are the voice of the user or sounds ofa pen switch provided on the pen device.

By means of such a configuration, the user can clearly show the starttime/end time of handwritten data through use of his/her own voice orsounds of a pen switch.

The invention also provides a personal identification method includingthe handwritten data input method, the personal identification methodcomprising:

a comparison verification step in which the user's signature data inputas handwritten data by the handwritten data input apparatus are verifiedagainst and compared with the reference data which are stored beforehandand serve as signature data pertaining to an authorized user, and thereis output authentication of identification when the handwritten data andthe signature data are determined to be a signature affixed by the sameperson.

By means of such a configuration, there is obtained a personalidentification apparatus having the same operation as that of the piecesof handwritten data input apparatus. Particularly, since signature datain the form of three-dimensional data are utilized, highly accuratepersonal identification can be achieved.

The invention also provides a handwritten data input apparatus foracquiring handwritten data written in midair by means of a user moving afinger in midair, comprising:

first camera means for acquiring first motion picture data pertaining tothe finger moved by the user;

second camera means for acquiring second motion picture data pertainingto the finger from an angle different from that of the first camerameans; and

coordinate computation means for determining three-dimensionalcoordinates of the finger in three-dimensional space on the basis of thesets of first and second motion picture data and determining time-seriesdata pertaining to the coordinates.

By means of such a configuration, the user can clearly show the starttime/end time of handwritten data through use of his/her own voice orsounds of a pen switch.

The handwritten data input apparatus of the invention is furthercharacterized in that the finger is provided with a marker, and thecoordinate computation means determines time-series data pertaining tocoordinates of the marker of the finger.

By means of such a configuration, the marker facilitates computation ofcoordinates of the finger.

The invention also provides a handwritten data input method foracquiring handwritten data written in midair by means of a user moving afinger in midair, comprising:

a first acquisition step of acquiring first motion picture datapertaining to the finger moved by the user;

a second acquisition step of acquiring second motion picture datapertaining to the finger from an angle different from that employed inthe first acquisition step; and

a coordinate computation step of determining three-dimensionalcoordinates of the finger in three-dimensional space on the basis of thesets of first and second motion picture data and determining time-seriescoordinate pertaining to the coordinates.

By means of such a configuration, the user can clearly show the starttime/end time of handwritten data through use of his/her own voice orsounds of a pen switch.

The handwritten data input method of the invention is furthercharacterized in that the finger is provided with a marker, and, in thecoordinate computation step, time-series data pertaining to coordinatesof the marker of the finger are determined.

By means of such a configuration, the marker facilitates computation ofcoordinates of the finger.

The handwritten data input apparatus of the invention is furthercharacterized in that the illumination means emits modulated light, andthe coordinate computation means computes coordinates of a portion ofthe motion picture data corresponding to the modulated light.

The handwritten data input method of the invention is furthercharacterized in that the illumination means emits modulated light, and,in the coordinate computation step, coordinates of a portion of themotion picture data corresponding to the modulated light are computed.

By means of the configuration of the invention, coordinates of a pendevice (or illumination means) can be determined readily by means ofselecting only modulated light.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of a personalidentification apparatus according to a preferred embodiment A-1 of theinvention;

FIG. 2 is a flowchart showing the flow of an operation of a personalidentification method according to the embodiment A-1;

FIG. 3 is a descriptive view of a pen device;

FIG. 4 is a descriptive view of computation of coordinates;

FIG. 5 is a descriptive view showing an example using a flat glassplate;

FIG. 6 is a descriptive view showing an example using display means;

FIG. 7 is a block diagram showing the configuration of a personalidentification apparatus according to a preferred embodiment B-1 of theinvention;

FIG. 8 is a flowchart showing the flow of an operation of a personalidentification method according to the embodiment B-1;

FIG. 9 is a descriptive view of a pen device;

FIG. 10 is a descriptive view showing the principle to be used fordetermining three-dimensional coordinates; and

FIG. 11 is a descriptive view showing an example using display means.

BEST MODES FOR IMPLEMENTING THE INVENTION

A first group of preferred embodiments of the invention will bedescribed hereinbelow by reference to the drawings.

Embodiment A-1

FIG. 1 is a block diagram showing the configuration of a personalidentification apparatus 10 according to a preferred embodiment A-1 ofthe invention. As illustrated, the personal identification apparatus 10comprises handwritten data input means 12 for enabling entry ofsignature data which are handwritten data pertaining to a user; andsignature matching means 14 which verifies the handwritten data againstpreviously-recorded signature data through comparison, to therebydetermine whether or not personal identification is authenticated.

The handwritten data input means 12 comprises camera means 20, andcoordinate computation means 22. The camera means 20 photographs animage of a pen device 30 to be moved in midair by a user and outputsimage data in the form of motion pictures. The coordinate computationmeans 22 computes time-series data pertaining to coordinates of the pentip of the pen device 30 from the set of image data. The coordinates arecoordinates on a screen of image data; that is, two-dimensional values.Time-series data pertaining to the coordinates represent handwrittendata written in midair by the user through use of the pen device 30.

When the user has written a signature in midair, the handwritten datacan be utilized as signature data which are to become an object ofpersonal identification.

The signature matching means 14 has comparison verification means 40 andstorage means 42.

The comparison verification means 40 verifies the handwritten data(signature data) output from the coordinate computation means 22 bymeans of comparison with reference data previously stored in the storagemeans 42. If the result of comparison shows that the handwritten dataare similar to the reference data to such an extent that the handwrittendata can be determined to be affixed by one and the same person, anauthentication result showing “Identification is authenticated” isoutput. Here, the expression “reference data” means signature datapreviously registered by the original user.

Detailed personal identification operation of the personalidentification apparatus 10; that is, an operation of the personalidentification method, will now be described by reference to a flowchartshown in FIG. 2.

In step S2-1, the user affixes a signature by means of moving the pendevice 30 in midair. FIG. 3 is a descriptive view of the pen device 30.As illustrated, illumination means 32 is provided at the tip end of thepen device 30. The illumination means 32 has an LED (light-emittingdiode) and a lamp and can illuminate light.

The pen device 30 has a button switch 34. The user can activate ordeactivate the illumination means 32, by means of actuating the buttonswitch 34 with a finger.

First, the user activates the illumination means 32 by use of the buttonswitch 34, thereby indicating commencement of a signing action. The userperforms a signing action while the illumination means 32 remainsilluminated. At a point in time when the signing action has beencompleted, the user actuates the button switch 34, thereby deactivatingthe illumination means 32. The deactivating operation enables indicationof the end of the signing action.

The pen device 30 having the illumination means 32 is utilized in theembodiment A-1 for two reasons; (1) for explicitly pointing out thestart and end of signing action in the manner as mentioned previously;and (2) for facilitating detection of coordinates of the pen tip. Theseactions will be discussed in detail later.

In step S2-2, the camera means 20 photographs the pen device 30 andoutputs resultant image data. The camera means 20 corresponds to a videocamera. The image data to be output are motion picture data of 120frames/sec or thereabouts. In this way, in step S2-3 motion picture dataare acquired.

A common video camera which outputs motion picture data of 15 to 30frames/sec or thereabouts can also be utilized, although accuracy ofpersonal identification is lowered slightly.

In step S2-3, the coordinate computation means 22 determines coordinatesof the pen means 30 on the screen on the basis of the image data (motionpicture data), thereby determining time-series data pertaining tocoordinates; i.e., signature data.

At this time, the coordinate computation means 22 seeks coordinates of aportion of the image data having a high brightness level. Specificallyspeaking, an area of the image data whose brightness level is greaterthan or equal to than a predetermined threshold level α is extracted,and the centroid of brightness of the area is determined. The positionof the centroid is deemed as the pen tip, and coordinates of the pen tipare determined. Every portion of the pen device 30 may be used fordetermining the position of centroid. In general, the “tip” of a pen iseasy to photograph, and hence determination of coordinates of the “pentip” is considered to be desirable.

In this way, in the embodiment A-1, the position of the pen device canbe readily determined from image data by use of the pen device 30 havingthe illumination means 32.

In the embodiment A-1, coordinates of the pen device 30 are determinedon the screen. This operation is intrinsically the equivalent ofprojecting the position of the pen device 30 onto a virtual plane inspace and determining coordinates of the point of projection.

FIG. 4 is a descriptive view of a virtual plane P1 to be used at thetime of calculation of coordinates. The plane P1 is parallel to thescreen, and both the plane P1 and the screen are perpendicular to theoptical axis of the camera means 20. A trail of the pen device 30 on thescreen appearing in the image data is similar to a trail produced whenthe locus of the pen device 30 in space stemming from the user havingperformed a signing action is projected onto the virtual plane P1. Thosetrails correspond to essentially the same data.

Consequently, determination of a horizontal coordinate (i.e., the Xcoordinate in FIG. 4) and a longitudinal coordinate (i.e., the Ycoordinate in FIG. 4) on the screen is substantially the equivalent ofdetermining data pertaining to coordinates of a trail on the plane P1produced when the locus of the pen device 30 stemming from the userhaving performed a signing action is projected onto the plate P1.

The coordinate computation means 22 determines a start time and a finishtime of signature data corresponding to time-series data, as follows.

First, a point in time when an area having a predetermined thresholdbrightness level or higher has appeared in image data is determined tobe a point in time when the user has started a signing action, andtime-series data originating from that point in time are taken assignature data.

Next, a point in time when the area having a predetermined thresholdbrightness level or higher has disappeared from the image data isdetermined to be a point in time when the user has completed the signingaction, and the time-series data that have been generated up to thatpoint in time are taken as signature data.

Since the coordinate computation means 22 performs such an operation,this means can readily determine the start and end of signature data inaccordance with the user's signing action described in connection withstep S2-1.

In step S2-4, the comparison verification means 40 verifies thethus-determined signature data by comparison with reference data. When aresult of comparison shows that the signature data are similar to thereference data to such an extent that both data are determined tooriginate from one and the same person, an authentication result showingthat “personal identification is authenticated” is output. In contrast,if the signature data are not similar to the reference data, anotherauthentication result showing that “personal identification is notauthenticated” is output The reference data are signature dataregistered in advance in the storage means 42 by the original user. Thereference data are also signature data pertaining to a signing actionperformed in midair beforehand by the original user.

Time-series data pertaining to coordinates produced when the user hasaffixed a signature in midair are time-series data pertaining tocoordinates of the pen tip, as in the case of signature data acquired byway of a related-art electronic tablet. A difference between those setsof time-series data lies only in the issue of whether the signature wasaffixed on the electronic tablet or in midair. Those data sets aresimilar to each other in terms of an intrinsic nature of a signature;that is, biometric data pertaining to an individual.

As mentioned above, the embodiment A-1 enables personal identificationon the basis of signature data written in midair by the user.

Here, the coordinate computation means 22 and the comparisonverification means 40 are preferably constituted of a computer; i.e., acombination of software and a processor for executing the software. Inthe case where high-speed authentication operation is especiallyrequired, the coordinate computation means and the comparisonverification means are preferably constituted of hardware, such as acustom-designed LSI. Moreover, the coordinate computation means 22 ispreferably constituted through use of a so-called video tracker.

The storage means 42 may be constituted of magnetic storage means, suchas a hard disk drive, a flexible disk, or the like; or may beconstituted of an optical disk, such as a CD-ROM, an MO, or the like. Inaddition, the storage means 42 may utilize semiconductor storage means,such as flash memory.

Embodiment A-2 (A Modification Employing a Different Method ofDetermining Start and End)

In the embodiment A-1, the coordinate computation means 22 determinesthe start time and end time of signing action from the activated ordeactivated state of the pen device 30.

However, the start time and end time may be determined by means ofanother technique.

For example, in a case where the user has left the pen device 30 at restin midair for a period of time longer than a predetermined period T1 andthen started moving the pen device 30, a point in time when movement ofthe pen is started is determined to be a start time of signing action.

In this case, when a time longer than the predetermined period time T1has elapsed since the pen device 30 become stationary, the coordinatecomputation means 22 preferably informs the user, by means of bellsounds, that a time longer than the period T1 has elapsed and that entryof a signature has become possible. Alternatively, voice output of amessage stating that “Please start affixing a signature” or the like isalso preferable. In addition, illumination of a predetermined light isalso desirable. By means of these signals, the user can ascertain that asigning action can be started as a result of lapse of the period T1.

When the user has left the pen device 30 at rest in midair for a periodof time longer than a predetermined period T2 when terminating thesigning action, a point in time when the pen device has startedremaining at rest is determined to be an end time of the signing action(i.e., a point in time when the signing action has been stopped).

In this case, when a period of time longer than the predetermined periodT2 has elapsed since the pen device 30 was laid to rest, the coordinatecomputation means 22 preferably informs the user of lapse of time, bymeans of bell sounds. Alternatively, voice output of a message statingthat “Acquisition of signature data has been completed” or the like isalso preferable. In addition, illumination of a predetermined light isalso desirable. By means of these signals, the user can ascertain thatthe signing action has been completed as a result of lapse of the periodT2.

With regard to whether or not the pen device 30 remains at rest, the pendevice is preferably determined to remain at rest when variations in thecoordinates of the pen device 30 are lower than a predetermined level.When a person lays the pen device 30 to rest in midair, somewhatfluctuations are usually considered to arise. Therefore, if a strictdefinition of “stationary” is applied to a determination, the pen deviceis determined to be “moving” even when only slight changes have arisenin the coordinates of the pen device, thereby rendering thedetermination inappropriate.

Therefore, a very small level is prepared. If movement of the pen deviceis smaller than this level, the pen device is preferably determined toremain “at rest.” Here, the word “movement” generally signifiesvariations in the coordinates of a pen across frames of an image.

Embodiment A-3 (A Modification of the Pen Device 30)

In the embodiment A-1, the user has performed the signing action inmidair through use of the pen device 30 having the illumination means32. The pen device 30 is illustrated in FIG. 3 as if assuming the samegeometry as that of an ordinary pen. However, the pen device may assumeany geometry, so long as the pen has the illumination means 32.

In the embodiment A-1 the pen device 30 is constructed as having theillumination means 32. However, the illumination means 32 may beomitted, so long as the pen device 30 can be distinguished fromsurroundings in an image. For instance, if the pen device 30 is given avery uncommon pattern or a coloration which is not in the environment,the pen tip can be distinguished on image data. In such a case, theillumination means 32 is not necessarily required.

When the illumination means 32 is not used, difficulty is encountered inrepresenting the start and end of signing action in the form of anilluminated/extinguished state of the illumination means 32. For thisreason, in a preferable configuration the start and end of a signingaction are indicated by means of laying the pen device 30 to rest for agiven period of time, as has been described in connection with theembodiment A-2. In another preferable configuration, the user indicatesthe start and end of a signing action through use of switching meansprovided separately on the personal identification apparatus 10.

Performing a signing action without use of the pen device 30 is alsopreferable. For example, if a highly-accurate image recognitiontechnique is employed, a user's finger or the like can be identified inan image. In this case, when the user has a color or geometry easy toidentify in an image, such as that of nail polish or a ring, the colorof the nail polish or the geometry of the ring is preferably identifiedthrough image recognition, and coordinate data pertaining to movement ofthe finger are preferably detected, thereby determining signature data.

Embodiment A-4 (A Handwritten Data Input Apparatus)

The embodiments A-1 through A-3 have described the personalidentification device and method using the handwritten data input means12. The handwritten data input means 12 enables entry of various sets ofhandwritten data other than signature data, as does the related-artelectronic tablet. Consequently, the handwritten data input means 12 ispreferably used as a handwritten data input apparatus in place of arelated-art electronic tablet, thereby constituting a device forinputting graphics or drawings.

Embodiment A-5 (A Guide Constituted of a Flat Plate)

The embodiments A-1 through A-4 have described a case where handwrittendata and signature data are written in midair. However, if a flat plateguide is provided, the user can more easily write handwritten data.

FIG. 5 shows such an example. As shown in the drawing, a flat glassplate 50 is transparent, and the user writes handwritten data on onesurface of the flat glass plate 50. At this time, the user can supportthe pen device 30 by means of presence of the flat glass plate 50.Hence, there is yielded an effect of rendering writing of handwrittendata more easy.

As shown in FIG. 5, the camera means 20 takes a photograph from the backof the flat glass plate 50. Use of the flat, transparent glass plate 50enables such a photographing operation. Here, the word “back” refers toa surface opposite to the surface of the plate on which the user iswriting handwritten data.

Although the flat glass plate 50 is employed, as a matter of course useof another material is possible, so long as the material forms atransparent flat plate.

Embodiment A-6 (A Finger or a Marker Placed on a Finger)

Although the embodiments A-1 through A-5 have used the pen device 30,use of a mere “finger” is also preferable. If the finger can beidentified in image data, handwritten data can be obtained astime-series data pertaining to coordinates on the same principle as thatdescribed above.

Alternatively, a marker is preferably attached to a finger. The markeris easy to identify in an image. Consequently, there is yielded aneffect of substantially facilitating determination of coordinates of thefinger by means of determining coordinates of the marker.

Here, the marker is a general concept regarding means having thefunction of making a subject conspicuous by means of reflecting light,such as a reflection tape, as well as regarding various types of paints,fluorescent paints, or luminous paints of conspicuous colors. The markeris easy to distinguish from the environment in an image. Hence,coordinates of the marker can be readily determined.

Embodiment A-7 (Start Time and End Time)

Although there has been described a technique of acquiring handwrittendata as time-series data pertaining to coordinates, in addition to thetechnique of making a determination on the basis of whether or not thepen device 30 is illuminated or extinguished, various other techniquesof determining the start time and end time of handwritten data areconceivable.

For example, a determination is preferably made by means of user's voiceor the sound of a pen switch. Further, the pen device 30 is preferablyprovided with a mechanism for issuing a predetermined sound, andpreferably the start and end of handwritten data with the predeterminedsound are explicitly indicated by means of the user activating themechanism.

As mentioned above, when a determination is made as to a start time andan end time by means of sound (or voice), the coordinate computationmeans 22 is preferably constructed so as to be supplied with such sounddata and to make a determination as to a start time and an end time onthe basis of the sound data. The reason for this is that the coordinatecomputation means 22 must ascertain a start time and an end time.

Embodiment A-8 (Display of Handwritten Data)

Entry of handwritten data has been described thus far. If the user canascertain the handwritten data input by himself/herself, conveniencewill be afforded to the user. FIG. 6 is a descriptive view showing theconfiguration of the personal identification apparatus 10 provided withdisplay means 60 for displaying the input handwritten data. The displaymeans 60 has the function of receiving time-series data pertaining tocoordinates and displaying respective coordinates. The display means 60can be embodied in the form of, e.g., a personal computer. Aliquid-crystal display screen or CRT may be used for the display means.

Even if coordinates have been acquired at coarse time intervals, asmooth display is obtained by means of interconnecting respectivecoordinates in chronological order through use of line segments anddisplaying the line segments at the time of a display operation.Preferably, interconnection of respective coordinates is displayed notthrough use of mere line segments but in the form of a graphicinterpolated by a curve such as a spline curve.

Simultaneous display of handwritten data and a handwriting operation;that is, real-time display of handwritten data, is generally preferable.A person tends to look at his/her hand while performing a writingoperation. However, a user could conceivably encounter difficulty inviewing a display screen while performing a writing action. For thisreason, conceivably many users desire to look at a display screen afterhaving finished performing a handwriting action.

Display of handwritten data in the manner as mentioned above enables theuser to ascertain the handwritten data entered by himself/herself, thusimproving the user's convenience.

Embodiment A-9 (Utilization of Modulated Light)

The illumination means 32 of the pen device 30 is higher in brightnesslevel than surroundings in an image. Hence, the pen device 30 can bereadily distinguished from surroundings and recognized in an image.

Modulated light is preferably used as light which is to originate fromthe illumination means 32.

For instance, light originating from the light emission means 32 isblinked at a frequency of 100 Hz or thereabouts. If only the light thatblinks at a frequency of 100 Hz is extracted from motion picture datathrough filtering, the position of the illumination means 32 can beclearly identified. As mentioned above, blinking the illumination means32 at predetermined cycles yields an effect particularly whenhandwritten data or signature data are input in surroundings of highbrightness level (e.g., outdoors in daylight, or a like situation). Ifsurroundings have a high brightness level, difficulty is encountered insetting the brightness level of the light originating from theillumination means 32 so as to become higher than that of thesurroundings. Hence, mere illumination of the illumination means 32 isalso conceived to impose difficulty in identifying the pen device 30.However, since a light source which repeatedly blinks at a cycle ofabout 100 Hz is rarely found in the natural world, adoption of modulatedlight enables easy identification of the pen device 30.

In relation to a filtering technique for extracting only modulated light(which blinks at a cycle of, e.g., 100 Hz), various filtering techniquesare available. Repeated blinking at a frequency of 100 Hz means, simply,that illuminating the light source for five milliseconds andextinguishing it for subsequent five milliseconds are repeated. Ifmotion picture data corresponding to a period of five millisecondsduring which the light source is extinguished are extracted from motionpicture data corresponding to a period of five milliseconds during whichthe light source is illuminated, the absolute value of brightness levelof the blinking illumination means 32 is substantially doubled. Incontrast, in relation to the surroundings whose brightness level remainssubstantially unchanged, essentially an identical value is subtractedfrom the motion picture data for illumination and from the motionpicture data for extinction. Hence, the value of image data pertainingto the surroundings approaches zero, whereas the absolute value of imagedata pertaining to an area corresponding to the illumination means 32 issubstantially doubled. If an area having a large absolute value isextracted from an image after the image data have been subjected to suchprocessing, the area of the illumination means 32 can be extractedreadily.

In the embodiment A-9, the coordinate computation means 22 performs theabove-described filtering operation. The coordinate computation means 22computes coordinates of the extracted area of the illumination means 32.

In this way, according to the embodiment A-9, modulated light is usedfor the light which is to originate from the illumination means 32. Evenwhen surroundings have a high brightness level, the illumination means32 can be distinguished in an image. Consequently, the position of thepen device 30 can be ascertained even in a bright location, and hencethe user can enter handwritten data and signature data even whenoutdoors in daylight.

A second group of preferred embodiments of the invention will bedescribed hereinbelow by reference to the drawings.

Embodiment B-1

FIG. 7 is a block diagram showing the configuration of a personalidentification apparatus 110 according to a preferred embodiment of theinvention. As illustrated, the personal identification apparatus 110comprises handwritten data input means 112 for enabling entry ofsignature data which are handwritten data pertaining to a user; andsignature matching means 114 which verifies the handwritten data againstpreviously-recorded signature data through comparison, to therebydetermine whether or not personal identification is authenticated.

The handwritten data input means 112 comprises first camera means 120 a,second camera means 120 b, and coordinate computation means 122.

The first camera means 120 a photographs an image of a pen device 130 tobe moved in midair (hereinafter also called “three-dimensional space”)by a user and outputs first image data in the form of motion pictures.The second camera means 120 b photographs, from a different angle, animage of the pen device 130 to be moved in midair by the user andoutputs second image data in the form of motion pictures.

The coordinate computation means 122 computes time-series datapertaining to coordinates of the pen tip of the pen device 130 fromthese sets of image data.

In the embodiment B-1, the coordinates are three-dimensional coordinatesin three-dimensional space. Time-series data pertaining to thethree-dimensional coordinates three-dimensionally represent handwrittendata written in midair (i.e., in three-dimensional space) by the userthrough use of the pen device 130.

In the embodiment B-1, user's handwritten data can be input asthree-dimensional coordinate data.

When the user has written a signature in midair, the handwritten datacan be utilized as signature data which are to become an object ofpersonal identification. Particularly in this case, the signature dataare three-dimensional data, and hence there can be realized personalidentification which is higher in accuracy than that obtained fromtwo-dimensional data.

The signature matching means 114 has comparison verification means 140and storage means 142.

The comparison verification means 140 verifies the handwritten data(signature data) output from the coordinate computation means 122 bymeans of comparison with reference data previously stored in the storagemeans 142. If the result of comparison shows that the handwritten dataare similar to the reference data to such an extent that the handwrittendata can be determined to be affixed by the same person, anauthentication result showing “Identification is authenticated” isoutput. Here, the expression “reference data” means signature datapreviously registered by the original user.

Detailed personal identification operation of the personalidentification apparatus 110; that is, an operation of the personalidentification method, will now be described by reference to a flowchartshown in FIG. 8.

In step S8-1, the user affixes a signature by means of moving the pendevice 130 in midair. FIG. 9 is a descriptive view of the pen device130. As illustrated, illumination means 132 is provided at the tip endof the pen device 130. The illumination means 132 has an LED(light-emitting diode) and a lamp and can emit light.

The pen device 130 has a button switch 134. The user can activate ordeactivate the illumination means 132 by means of actuating the buttonswitch 134 with a finger.

First, the user activates the illumination means 132 by use of thebutton switch 134, thereby indicating commencement of a signing action.The user performs a signing action while the illumination means 132remains illuminated. At a point in time when the signing action has beencompleted, the user actuates the button switch 134, thereby deactivatingthe illumination means 132. The deactivating operation enablesindication of the end of the signing action.

The pen device 30 having the illumination means 32 is utilized in theembodiment B-1 for two reasons: (1) for explicitly pointing out thestart and end of signing action in the manner as mentioned previously;and (2) for facilitating detection of coordinates of the pen device 130(particularly the pen tip thereof). These actions will be discussed indetail later.

In step S8-2, the first camera means 120 a photographs the pen device130 and outputs resultant image data (first image data). The camerameans 120 corresponds to a video camera. The first image data to beoutput are motion picture data of 120 frames/sec or thereabouts(hereinafter often referred to also as “first motion picture data”). Inthis way, instep S8-3 motion picture data are acquired.

In the embodiment B-1, acquisition of handwritten data is performed byuse of the second camera means 120 b, as well as by use of the firstcamera 120 a. More specifically, the second camera means 120 bphotographs the pen device 130 from an angle different from that inwhich the first camera means 120 a photographs and outputs image data(second image data). The second image data are also motion picture dataof about 120 frames/sec (hereinafter often referred to also as “secondmotion picture data”). In this way, in step S8-3 motion picture data areacquired from two different angles.

A common video camera which outputs motion picture data of 15 to 30frames/sec or thereabouts can also be utilized, although accuracy ofpersonal identification is lowered slightly.

In step S8-3, the coordinate computation means 122 determinescoordinates of the pen means 130 in three-dimensional space; that is,determines three-dimensional coordinates, on the basis of the sets ofimage data (the first and second sets of motion picture data). Further,time-series data pertaining to coordinates; i.e., signature data, aredetermined by means of sequential computation of the three-dimensionalcoordinates.

At this time, the coordinate computation means 122 seeks coordinates ofa portion of the image data having a high brightness level. Specificallyspeaking, an area of the image data whose brightness level is greaterthan or equal to than a predetermined threshold level α is extracted,and the centroid of brightness of the area is determined. The positionof the centroid is deemed as the pen tip, and coordinates of the pen tipare determined. Every portion of the pen device 130 may be used fordetermining the position of the centroid. A traditional signatureinvolving an action of affixing a signature on paper with a pen isperformed through use of a pen “tip,” and hence determination ofcoordinates of the “pen tip” is considered to be desirable in terms ofproviding a familiar situation.

In this way, in the embodiment B-1, the position of the pen device 130can be readily determined from image data by use of the pen device 130having the illumination means 132.

The position of the pen device 130 can be determined three-dimensionallyby use of the first and second motion picture data acquired fromdifferent angles. More specifically, three-dimensional coordinates ofthe pen device 130 can be determined.

FIG. 10 is a descriptive view showing a principle to be used fordetermining three-dimensional coordinates.

In the illustration, a plane perpendicular to the angle from which thefirst motion picture data have been photographed is represented by P1,and a plane perpendicular to the angle from which the second motionpicture data have been photographed is represented by P2. An eyepoint V1is set for the plane P1, and an eyepoint V2 is set for the plane P2. Aline of sight is extended from the eyepoint V1 to the position of thepen device on the resultant image, and another line of sight is extendedfrom the eyepoint V2 to the position of the pen device on anotherresultant image. Coordinates of an intersection of these lines of sightare computed, and the thus-computed coordinates are taken as coordinatesof the pen device 130.

Since the thus-photographed image data are motion picture data, theposition of the pen device 130 on the image changes momentarily.Time-series data pertaining to coordinates of the pen device 130 aredetermined by means of sequentially determining coordinates of the pendevice 130 so as to follow the variations.

In this way, signature data pertaining to the user can be obtained asthree-dimensional time-series data.

The coordinate computation means 122 determines a start time and afinish time of signature data corresponding to time-series data, asfollows.

First, a point in time when an area having a predetermined thresholdbrightness level α or higher has appeared in image data is determined tobe a point in time when the user has started a signing action, andtime-series data originating from that point in time are taken assignature data.

A point in time when the area having a predetermined thresholdbrightness level α or higher has disappeared from the image data isdetermined to be a point in time when the user has completed the signingaction, and the time-series data that have been generated up to thatpoint in time are taken as signature data.

Since the coordinate computation means 122 performs such an operation,this means can readily determine the start and end of signature data inaccordance with the user's signing action described in connection withstep S8-1. In other words, when the user illuminates the illuminationmeans 132 of the pen device 130 by means of actuating the button switch134, the coordinate computation means 122 detects the illuminationoperation and determines that a signing action has been started. Whenthe user extinguishes the illumination means 132 of the pen device 130by means of actuating the button switch 134, the coordinate computationmeans 122 detects the extinction operation and determines that thesigning operation has been completed.

In step S8-4, the comparison verification means 140 verifies thethus-determined signature data by comparison with reference data. When aresult of comparison shows that the signature data are similar to thereference data to such an extent that both data are determined tooriginate from the same person, an authentication result showing that“identification is authenticated” is output. In contrast, if thesignature data are not similar to the reference data, anotherauthentication result showing that “Identification is not authenticated”is output. The reference data are signature data registered in advancein the storage means 142 by the original user. The reference data arealso signature data pertaining to a signing action performed in midairbeforehand by the original user.

Time-series data pertaining to coordinates produced when the user hasaffixed a signature in midair are time-series data pertaining tothree-dimensional coordinates in midair. Consequently, the time-seriesdata are three-dimensional data, as in the case of signature dataincluding a writing pressure acquired by way of a related-art electronictablet to be used for detecting a writing pressure. Therefore, accuracysimilar to that obtained in the related art is considered to be achievedwhen three-dimensional data are utilized for signature matching.

As mentioned above, the embodiment B-1 enables personal identificationon the basis of signature data written in midair by the user.

Here, the coordinate computation means 122 and the comparisonverification means 140 are preferably constituted of a computer; i.e., acombination of software and a processor for executing the software. Inthe case where high-speed authentication operation is especiallyrequired, the coordinate computation means and the computationverification means are preferably constituted of hardware, such as acustom-designed LSI. Moreover, the coordinate computation means 122 ispreferably constituted through use of a so-called video tracker.

The storage means 142 may be constituted of magnetic storage means, suchas a hard disk drive, a flexible disk, or the like; or may beconstituted of an optical disk, such as a CD-ROM, an MO, or the like. Inaddition, the storage means 142 may utilize semiconductor storage means,such as flash memory.

Embodiment B-2 (A Modification Employing a Different Method ofDetermining Start and End)

In the embodiment B-1, the coordinate computation means 122 determinesthe start time and end time of signing action from the activated ordeactivated state of the pen device 130.

However, the start time and end time may be determined by means ofanother technique.

For example, in a case where the user has left the pen device 130 atrest in midair for a period of time longer than a predetermined periodT1 and then started moving the pen device 130, a point in time whenmovement of the pen is started is determined to be a start time ofsigning action.

In this case, when a time longer than the predetermined period time T1has elapsed since the pen device 130 has become stationary, thecoordinate computation means 122 preferably informs the user, by meansof bell sounds, that a time longer than the period T1 has elapsed andthat entry of a signature has become possible. Alternatively, voiceoutput of a message stating “Please start affixing a signature” or thelike is also preferable. In addition, illumination of a predeterminedlight is also desirable. By means of these signals, the user canascertain that a signing action can be started as a result of lapse ofthe period T1.

When the user has left the pen device 130 at rest in midair for a periodof time longer than a predetermined period T2 when terminating thesigning action, a point in time when the pen device has startedremaining at rest is determined to be an end time of the signing action(i.e., a point in time when the signing action has been stopped).

In this case, when a period of time longer than the predetermined periodT2 has elapsed since the pen device 130 has been laid to rest, thecoordinate computation means 122 preferably informs the user of lapse oftime, by means of bell sounds. Alternatively, voice output of a messagestating that “Acquisition of signature data has been completed” or thelike is also preferable. In addition, illumination of a predeterminedlight is also desirable. By means of these signals, the user canascertain that the signing action has been completed as a result oflapse of the period T2.

With regard to whether or not the pen device 130 remains at rest, thepen device is preferably determined to remain at rest when variations inthe coordinates of the pen device 130 are lower than a predeterminedlevel. When a person lays the pen device 130 to rest in midair, somefluctuations are usually considered to arise. Therefore, if a strictdefinition of “stationary” is applied to a determination, the pen deviceis determined to be “moving” even when only slight changes have arisenin the coordinates of the pen device, thereby rendering thedetermination inappropriate.

Therefore, a very small level is prepared. If movement of the pen deviceis smaller than this level, the pen device is preferably determined toremain “at rest.” Here, the word “movement” generally signifiesvariations in the coordinates of a pen across frames of an image.

Embodiment B-3 (A Modification of the Pen Device 130)

In the embodiment B-1, the user has performed the signing action inmidair through use of the pen device 130 having the illumination means132. The pen device 130 is illustrated in FIG. 9 as if assuming the samegeometry as that of an ordinary pen. However, the pen device may assumeany geometry, so long as the pen has the illumination means 132.

In the embodiment B-1, the pen device 130 is constructed as having theillumination means 132. However, the illumination means 132 may beomitted, so long as the pen device 130 can be distinguished fromsurroundings in an image. For instance, if the pen device 130 is given avery uncommon pattern or a coloration which is not in the environment,the pen tip can be distinguished on image data. In such a case, theillumination means 132 is not necessarily required.

When the illumination means 132 is not used, difficulty is encounteredin representing the start and end of signing action in the form of anilluminated/extinguished state of the illumination means 132. For thisreason, in a preferable configuration the start and end of a signingaction are indicated by means of laying the pen device 130 to rest for agiven period of time, as has been described in connection with theembodiment B-2. In another preferable configuration, the user indicatesthe start and end of a signing action through use of switching meansprovided separately on the personal identification apparatus 110.

Performing a signing action without use of the pen device 130 is alsopreferable. For example, if a highly-accurate image recognitiontechnique is employed, a user's finger or the like can be identified inan image. In this case, when the user has a color or geometry easy toidentify in an image, such as that of nail polish or a ring, the colorof the nail polish or the geometry of the ring is preferably identifiedthrough image recognition, and coordinate data pertaining to movement ofthe finger are preferably detected, thereby determining signature data.

Embodiment B-4 (A Handwritten Data Input Apparatus)

The embodiments B-1 through B-3 have described the personalidentification device and method using the handwritten data input means112. The handwritten data input means 112 enables entry of various setsof handwritten data other than signature data, as does the related-artelectronic tablet. Consequently, the handwritten data input means 112 ispreferably used as a handwritten data input apparatus in place of arelated-art electronic tablet, thereby constituting a device forinputting graphics or drawings.

Embodiment B-5 (A Finger or a Marker Placed on a Finger)

Although the embodiments B-1 through B-4 have used the pen device 130,use of a mere “finger” is also preferable. If the finger can beidentified in image data, handwritten data can be obtained astime-series data pertaining to coordinates on the same principle as thatdescribed above.

Alternatively, a marker is preferably attached to a finger. The markeris easy to identify in an image. Consequently, there is yielded aneffect of substantially facilitating determination of coordinates of thefinger by means of determining coordinates of the marker.

Here, the marker is a general concept regarding means having thefunction of making a subject conspicuous by means of reflecting light,such as a reflection tape, as well as regarding various types of paints,fluorescent paints, or luminous paints of conspicuous colors. The markeris easily distinguished from the environment in an image. Hence,coordinates of the marker can be readily determined.

Embodiment B-6 (Start Time and End Time)

Although there has been described a technique of acquiring handwrittendata as time-series data pertaining to coordinates, in addition to thetechnique of making a determination on the basis of whether or not thepen device 130 is illuminated or extinguished, various other techniquesof determining the start time and end time of handwritten data areconceivable.

For example, a determination is preferably made by means of user's voiceor the sound of a pen switch. Further, the pen device 130 is preferablyprovided with a mechanism for issuing a predetermined sound, andpreferably the start and end of handwritten data with the predeterminedsound are explicitly indicated by means of the user activating themechanism.

As mentioned above, when a determination is made as to a start time andan end time by means of sound (or voice), the coordinate computationmeans 122 is preferably constructed so as to be supplied with such sounddata and to make a determination as to a start time and an end time onthe basis of the sound data. The reason for this is that the coordinatecomputation means 122 must ascertain a start time and an end time.

Embodiment B-7 (Display of Handwritten Data)

Entry of handwritten data has been described thus far. If the user canascertain the handwritten data input by himself/herself, conveniencewill be afforded to the user. FIG. 11 is a descriptive view showing theconfiguration of the personal identification apparatus 110 provided withdisplay means 160 for displaying the input handwritten data. The displaymeans 160 has the function of receiving time-series data pertaining tocoordinates and displaying respective coordinates.

More accurately, the display is a projection drawing (two-dimensionaldisplay) which is produced by means of projecting three-dimensionalcoordinates on a two-dimensional plane. In the display, an eyepoint canbe preferably moved in accordance with a user's choice, or an object canbe preferably rotated.

The display means 160 can be embodied in the form of, e.g., a personalcomputer. A liquid-crystal display screen or CRT may be used for thedisplay means.

Even if coordinates have been acquired at coarse time intervals, asmooth display will be obtained by means of interconnecting respectivecoordinates in chronological order through use of line segments anddisplaying the line segments at the time of a display operation.Preferably, interconnection of respective coordinates is displayed notthrough use of mere line segments but in the form of a graphicinterpolated by a curve such as a spline curve.

Simultaneous display of handwritten data and a handwriting operation;that is, real-time display of handwritten data, is generally preferable.However, displaying the entirety of handwritten data after completion ofhandwriting action is also preferable.

Display of handwritten data in the manner as mentioned above enables theuser to ascertain the handwritten data entered by himself/herself, thusenhancing the user's convenience.

Embodiment B-8 (Utilization of Modulated Light)

The illumination means 132 of the pen device 130 is higher in brightnesslevel than surroundings in an image. Hence, the pen device 130 can bereadily distinguished from surroundings and recognized in an image.

Modulated light is preferably used as light which is to originate fromthe illumination means 132.

For instance, light originating from the light emission means 132 isblinked at a frequency of 100 Hz or thereabouts. If only the light thatblinks at a frequency of 100 Hz is extracted from motion picture datathrough filtering, the position of the illumination means 132 can beclearly identified. As mentioned above, blinking the illumination means132 at predetermined cycles yields an effect particularly whenhandwritten data or signature data are input in surroundings of highbrightness level (e.g., outdoors in daylight, or a like situation). Ifsurroundings have a high brightness level, difficulty is encountered insetting the brightness level of the light originating from theillumination means 132 so as to become higher than that of thesurroundings. Hence, mere illumination of the illumination means 132 isalso conceived to impose difficulty in identifying the pen device 130.However, since a light source which repeatedly blinks at a cycle ofabout 100 Hz is rarely present in the natural world, adoption ofmodulated light enables easy identification of the pen device 130.

In relation to a filtering technique for extracting only modulated light(which blinks at a cycle of, e.g., 100 Hz), various filtering techniquesare available. Repeated blinking at a frequency of 100 Hz means simplythat illuminating the light source for five milliseconds andextinguishing it for subsequent five milliseconds are repeated. Ifmotion picture data corresponding to a period of five millisecondsduring which the light source is extinguished are extracted from motionpicture data corresponding to a period of five milliseconds during whichthe light source is illuminated, the absolute value of brightness levelof the blinking illumination means 132 is substantially doubled. Incontrast, in relation to the surroundings whose brightness level remainssubstantially unchanged, essentially an identical value is subtractedfrom the motion picture data for illumination and from the motionpicture data for extinction. Hence, the value of image data pertainingto the surroundings approaches zero, whereas the absolute value of imagedata pertaining to an area corresponding to the illumination means 132is substantially doubled. If an area having a large absolute value isextracted from an image after the image data have been subjected suchprocessing, the area of the illumination means 132 can be extractedreadily.

In the embodiment B-8, the coordinate computation means 122 performs theabove-described filtering operation. The coordinate computation means122 computes coordinates of the extracted area of the illumination means132.

In this way, according to the embodiment B-8, modulated light is usedfor the light which is to originate from the illumination means 132.Even when surroundings have a high brightness level, the illuminationmeans 132 can be distinguished in an image. Consequently, the positionof the pen device 130 can be ascertained even in a bright location, andhence the user can enter handwritten data and signature data evenoutdoors in daylight.

As mentioned above, according to the first group of embodiments of theinvention, the handwritten data written in midair by the user can beinput. Hence, the handwritten data input apparatus can be miniaturizedas compared with a case where an electronic tablet requiring a certainarea is used. Consequently, even a compact electronic device, such as aportable cellular phone, can be provided with a handwritten data inputfunction.

If the pen device is equipped with illumination means, coordinates of apen tip become easy to detect, thus improving the accuracy of input ofhandwritten data.

If signature data to be used for personal identification are input byuse of such a handwritten data input apparatus, personal identificationcan be carried out by means of a compact electronic device.

Since the start time and end time of signature data are represented bythe illuminated/extinguished state of the illumination means, adetermination can be readily made as to the start or end of acquisitionof signature data.

If the start and end of input of handwritten data are determined bymeans of detecting a state in which the pen device remains at rest for agiven period of time or more, provision of illumination means isobviated. In this case, the user can readily ascertain a start time/endtime by means of being informed of a determination result with use ofsounds at a point in time when the start or end has been determined.

If the personal identification device/method is configured byutilization of the handwritten data input apparatus/method, a personalidentification device/method which yields a similar effect can beconstructed.

If the handwritten data input apparatus is constructed such thathandwritten data are written along a predetermined flat plate, ahandwriting operation can be performed smoothly.

Since the invention is constructed such that the start time/end time ofhandwritten data is instructed by means of the user's voice or the soundof a pen switch, the user can explicitly show the start time/end time,thereby enabling smooth entry of handwritten data or signing action.

Since there is provided means for displaying input handwritten data, theuser can check the handwritten data written by himself/herself.Therefore, safe entry of handwritten data is possible.

The invention is constructed such that handwritten data are written byuse of a user's finger without use of a pen device, thereby obviating anecessity for preparing a pen device. Particularly, if a finger isprovided with a marker, coordinates of the finger can be determinedreadily by means of determining coordinates of the marker.

The invention utilizes modulated light, thereby enabling easyidentification of the light originating from the pen device.Consequently, even when surroundings are bright, entry of handwrittendata and signature data can be effected.

As has bee described, according to the second group of embodiments ofthe invention, the handwritten data written in midair by the user can beinput as time-series data pertaining to three-dimensional coordinates.

Particularly, according to the invention, since handwritten data arerepresented by three-dimensional coordinates in midair, the accuracy ofsignature verification can be improved.

By means of providing the pen device with the illumination means,coordinates of the pen device become easy to detect, thus improving theaccuracy of input of handwritten data.

So long as signature data to be utilized for personal identification areinput by use of such a handwritten data input apparatus, personalidentification can be performed by means of a compact electronic device.

Since the start time and end time of signature data are represented byan illuminated/extinguished state of the illumination means, adetermination can be readily made as to the start/end of acquisition ofsignature data.

If the start and end of input of handwritten data are determined bymeans of detecting a state in which the pen device remains at rest for agiven period of time or more, there is obviated a necessity forproviding the pen device with illumination means. Particularly in thiscase, the user can readily ascertain a start time/end time by means ofbeing informed of a determination result by use of sounds at a point intime when the start or end has been determined.

If the personal identification device/method is configured byutilization of the handwritten data input apparatus/method, a personalidentification device/method which yields a similar effect can beconstructed.

Since the invention is constructed such that the start time/end time ofhandwritten data is instructed by means of the user's voice or the soundof a pen switch, the user can explicitly show the start time/end time,thereby enabling smooth entry of handwritten data or signing action.

Since there is provided means for displaying input handwritten data, theuser can check the handwritten data written by himself/herself.Therefore, safe entry of handwritten data is possible.

The invention is constructed such that handwritten data are written byuse of a user's finger without use of a pen device, thereby obviating anecessity for preparing a pen device. Particularly, if a finger isprovided with a marker, coordinates of the finger can be determinedreadily by means of determining coordinates of the marker.

The invention utilizes modulated light, thereby enabling easyidentification of light originating from the pen device. Consequently,even when surroundings are bright, entry of handwritten data andsignature data can be effected.

1. A handwritten data input apparatus which acquires handwritten datawritten in midair by means of a user moving a pen device in midair, theapparatus comprising: camera means for acquiring motion picture datapertaining to a pen device moved by the user; and coordinate computationmeans which determines time-series data pertaining to coordinates of apen tip on a screen of the motion picture data, on the basis of motionpicture data pertaining to the pen device acquired by the camera means,wherein the coordinate computation means computes coordinates of the pendevice on the screen from a start time in which the pen device hasstarted moving on the screen from a stationary state longer than apredetermined first period until a stop time in which the pen devicestops moving as a result of having entered a stationary state longerthan a predetermined second period.
 2. The handwritten data inputapparatus according to claim 1, wherein the coordinate computation meansdetermines the pen device to be at rest when a change in coordinates ofthe pen device is smaller than a predetermined level.
 3. The handwrittendata input apparatus according to claim 1, wherein the coordinatecomputation means informs the user of commencement of input ofhandwritten data when the pen device is determined to be at rest for aperiod longer than the first period and informs the user of terminationof input of handwritten data when the pen device is determined to be atrest for a period longer than the second period.
 4. The handwritten datainput apparatus according to claim 3, wherein the coordinate computationmeans informs the user of commencement of input of handwritten data bymeans of sound or informs termination of input of handwritten data bymeans of sound.
 5. A handwritten data input apparatus which acquireshandwritten data written in midair by means of a user moving a pendevice in midair, the apparatus comprising: camera means for acquiringmotion picture data pertaining to a pen device moved by the user; andcoordinate computation means which determines time-series datapertaining to coordinates of a pen tip on a screen of the motion picturedata, on the basis of motion picture data pertaining to the pen deviceacquired by the camera means, wherein the coordinate computation meansrecognizes a start time and end time of computation of handwritten databy means of external sounds.
 6. The handwritten data input apparatusaccording to claim 5, wherein the sounds are the voice of the user orsounds of a pen switch provided on the pen device.
 7. A handwritten datainput method by which handwritten data written in midair are acquired bymeans of a user moving a pen device in midair, the method comprising: anacquisition step of acquiring motion picture data pertaining to a pendevice moved by the user; and a coordinate computation step ofdetermining time-series data pertaining to coordinates of a pen tip on ascreen of the motion picture data, on the basis of the acquired motionpicture data pertaining to the pen device, wherein there are computed,in the coordinate computation step, coordinates of the pen device on thescreen from a start time in which the pen device has started moving onthe screen from a stationary state longer than a predetermined firstperiod until a stop time in which the pen device stops moving as aresult of having entered a stationary state longer than a predeterminedsecond period.
 8. The handwritten data input method according to claim7, wherein, in the coordinate computation step, the pen device isdetermined to be at rest when a change in coordinates of the pen deviceis smaller than a predetermined level.
 9. The handwritten data inputmethod according to claim 7, wherein, in the coordinate computationstep, the user is informed of commencement of input of handwritten datawhen the pen device is determined to be at rest for a period longer thanthe first period and is informed of termination of input of handwrittendata when the pen device is determined to be at rest for a period longerthan the second period.
 10. The handwritten data input method accordingto claim 9, wherein, in the coordinate computation step, the user isinformed of commencement of input of handwritten data by means of soundor informs termination of input of handwritten data by means of sound.11. A handwritten data input method by which handwritten data written inmidair are acquired by means of a user moving a pen device in midair,the method comprising: an acquisition step of acquiring motion picturedata pertaining to a pen device moved by the user; and a coordinatecomputation step of determining time-series data pertaining tocoordinates of a pen tip on a screen of the motion picture data, on thebasis of the acquired motion picture data pertaining to the pen device,wherein, in the coordinate computation step, a start time and end timeof computation of handwritten data are recognized by means of externalsounds.
 12. The handwritten data input method according to claim 11,wherein the sounds are the voice of the user or sounds of a pen switchprovided on the pen device.
 13. A handwritten data input apparatus whichacquires handwritten data written in midair by means of a user moving apen device in midair, the apparatus comprising: first camera means foracquiring first motion picture data pertaining to the pen device movedby the user; second camera means for acquiring second motion picturedata pertaining to the pen device from an angle different from that ofthe first camera means; and coordinate computation means for determiningthree-dimensional coordinates of the pen device in three-dimensionalspace on the basis of the sets of first and second motion picture dataand determining time-series data pertaining to the coordinates, whereinthe coordinate computation means computes coordinates of the pen deviceon the screen from a start time in which the pen device has startedmoving on the screen from a stationary state longer than a predeterminedfirst period until a stop time in which the pen device stops moving as aresult of having entered a stationary state longer than a predeterminedsecond period.
 14. The handwritten data input apparatus according toclaim 13, wherein the coordinate computation means determines the pendevice to be at rest when a change in coordinates of the pen device issmaller than a predetermined level.
 15. The handwritten data inputapparatus according to claim 13, wherein the coordinate computationmeans informs the user of commencement of input of handwritten data whenthe pen device is determined to be at rest for a period longer than thefirst period and informs the user of termination of input of handwrittendata when the pen device is determined to be at rest for a period longerthan the second period.
 16. The handwritten data input apparatusaccording to claim 15, wherein the coordinate computation means informsthe user of commencement of input of handwritten data by means of soundor informs termination of input of handwritten data by means of sound.17. A handwritten data input apparatus which acquires handwritten datawritten in midair by means of a user moving a pen device in midair, theapparatus comprising: first camera means for acquiring first motionpicture data pertaining to the pen device moved by the user; secondcamera means for acquiring second motion picture data pertaining to thepen device from an angle different from that of the first camera means;and coordinate computation means for determining three-dimensionalcoordinates of the pen device in three-dimensional space on the basis ofthe sets of first and second motion picture data and determiningtime-series data pertaining to the coordinates, wherein the coordinatecomputation means recognizes a start time and end time of computation ofhandwritten data by means of external sounds.
 18. A handwritten datainput apparatus which acquires handwritten data written in midair bymeans of a user moving a pen device in midair, the apparatus comprising:first camera means for acquiring first motion picture data pertaining tothe pen device moved by the user; second camera means for acquiringsecond motion picture data pertaining to the pen device from an angledifferent from that of the first camera means; and coordinatecomputation means for determining three-dimensional coordinates of thepen device in three-dimensional space on the basis of the sets of firstand second motion picture data and determining time-series datapertaining to the coordinates, wherein the sounds are the voice of theuser or sounds of a pen switch provided on the pen device.
 19. Ahandwritten data input method by which handwritten data written inmidair are acquired by means of a user moving a pen device in midair,the method comprising: a first acquisition step of acquiring firstmotion picture data pertaining to a pen device moved by the user from afirst angle; a second acquisition step of acquiring second motionpicture data pertaining to a pen device moved by the user from a secondangle different from the first angle; and a coordinate computation stepof determining three-dimensional coordinates of the pen device on thebasis of the acquired sets of first and second motion picture datapertaining to the pen device and determining time-series data pertainingto the coordinates, wherein, in the coordinate computation step,coordinates of the pen device on the screen are computed from a starttime in which the pen device has started moving on the screen from astationary state longer than a predetermined first period until a stoptime in which the pen device stops moving as a result of having entereda stationary state longer than a predetermined second period.
 20. Thehandwritten data input method according to claim 19, wherein, in thecoordinate computation step, the pen device is determined to be at restwhen a change in coordinates of the pen device is smaller than apredetermined level.
 21. The handwritten data input method according toclaim 19, wherein, in the coordinate computation step, the user isinformed of commencement of input of handwritten data when the pendevice is determined to be at rest for a period longer than the firstperiod and is informed of termination of input of handwritten data whenthe pen device is determined to be at rest for a period longer than thesecond period.
 22. The handwritten data input method according to claim21, wherein, in the coordinate computation step, the user is informed ofcommencement of input of handwritten data by means of sound or informstermination of input of handwritten data by means of sound.
 23. Ahandwritten data input method by which handwritten data written inmidair are acquired by means of a user moving a pen device in midair,the method comprising: a first acquisition step of acquiring firstmotion picture data pertaining to a pen device moved by the user from afirst angle; a second acquisition step of acquiring second motionpicture data pertaining to a pen device moved by the user from a secondangle different from the first angle; and a coordinate computation stepof determining three-dimensional coordinates of the pen device on thebasis of the acquired sets of first and second motion picture datapertaining to the pen device and determining time-series data pertainingto the coordinates, wherein, in the coordinate computation step, a starttime and an end time of computation of handwritten data are recognizedby means of external sounds.
 24. A handwritten data input method bywhich handwritten data written in midair are acquired by means of a usermoving a pen device in midair, the method comprising: a firstacquisition step of acquiring first motion picture data pertaining to apen device moved by the user from a first angle; a second acquisitionstep of acquiring second motion picture data pertaining to a pen devicemoved by the user from a second angle different from the first angle;and a coordinate computation step of determining three-dimensionalcoordinates of the pen device on the basis of the acquired sets of firstand second motion picture data pertaining to the pen device anddetermining time-series data pertaining to the coordinates, wherein thesounds are the voice of the user or sounds of a pen switch provided onthe pen device.